Multi-Task Learning by Deep Collaboration and Application in Facial Landmark Detection

Convolutional neural networks (CNN) have become the most successful and popular approach in many visionrelated domains. While CNNs are particularly well-suited for capturing a proper hierarchy of concepts from real-world images, they are limited to domains where data is abundant. Recent attempts have looked into mitigating this data scarcity problem by casting their original single-task problem into a new multitask learning (MTL) problem. The main goal of this inductive transfer mechanism is to leverage domain-specific information from related tasks, in order to improve generalization on the main task. While recent results in the deep learning (DL) community have shown the promising potential of training task-specific CNNs in a soft parameter sharing framework, integrating the recent DL advances for improving knowledge sharing is still an open problem. In this paper, we propose the Deep Collaboration Network (DCNet), a novel approach for connecting task-specific CNNs in a MTL framework. We define connectivity in terms of two distinct non-linear transformations. One aggregates taskspecific features into global features, while the other merges back the global features with each task-specific network. Based on the observation that task relevance depends on depth, our transformations use skip connections as suggested by residual networks, to more easily deactivate unrelated task-dependent features. To validate our approach, we employ facial landmark detection (FLD) datasets as they are readily amenable to MTL, given the number of tasks they include. Experimental results show that we can achieve up to 24.31% relative improvement in failure rate over other state-of-the-art MTL approaches. We finally perform an ablation study showing that our approach effectively allows knowledge sharing, by leveraging domainspecific features at particular depths from tasks that we know are related.